Indexing Reduced Dimensionality Spaces Using Single DimensionalIndexesHeng

The dimensionality curse has greatly aaected the scalability of high-dimensional indexes. A well known approach to improving the indexing performance is dimensionality reduction before indexing the data in the reduced-dimensionality space. However, the reduction may cause loss of distance information when the data set is not globally correlated. To reduce loss of information and degradation of search quality, cluster based dimensionality reduction should be used instead. In this paper, we present an adaptive local dimensionality reduction (LDR) technique which rst identiies eeective clusters based on Mahalanobis distance, and for each cluster, performs local dimensionality reduction. The data points in each cluster of the reduced-dimensionality space are then transformed into single distance values with reference to the centroid of the cluster, and indexed using a single dimensional index for nearest neighbor search. Unlike an existing LDR technique which uses an index for each cluster, we use one single B +-tree for the whole data set. Extensive performance studies using both real and synthetic data show that the method achieves higher precision compared to existing global dimensionality reduction and local dimensionality reduction methods, and is more eecient in terms of query performance.